Benchmarking online sequence-to-sequence and character-based handwriting recognition from IMU-enhanced pens

Abstract Handwriting is one of the most frequently occurring patterns in everyday life and with it comes challenging applications such as handwriting recognition, writer identification signature verification. In contrast to offline HWR that only uses spatial information (i.e., images), online richer spatio-temporal trajectory data or inertial data). While there exist many datasets, are little available for development OnHWR methods on paper requires hardware-integrated pens. This presents benchmark models real-time sequence-to-sequence learning single character-based recognition. Our recorded by a sensor-enhanced ballpoint pen, yielding sensor streams from triaxial accelerometers, gyroscope, magnetometer force at 100 Hz. We propose variety datasets including equations words both writer-dependent writer-independent tasks. allow comparison between classical tablets provide an evaluation seq2seq using recurrent temporal convolutional networks transformers combined connectionist classification (CTC) loss cross-entropy (CE) losses. network BiLSTMs outperforms transformer-based architectures, par InceptionTime sequence-based tasks yields better results compared 28 state-of-the-art techniques. Time-series augmentation improve task, we show CE variants can task. implementations together large techniques novel serve baseline future research area paper.